This invention relates to a boundary line detecting apparatus, image processing apparatus and non-boundary line detecting apparatus. More specifically, it relates to a boundary detecting apparatus which can detect the boundary line of different objects in contact which appear to be one object due to the fact that their structural consistencies are identical or substantially identical, to an image processing apparatus which can obtain a three-dimensional image of a fetus without interference from a uterus wall in contact with the fetus, to a non-boundary line detecting apparatus which can detect a non-boundary line of different objects in contact which appear to have a boundary as their structural consistencies are largely different, but which are actually part of the same structure, and to an image forming apparatus which can depict a liver so that an invasive cancer on the liver surface can be identified.
In an ultrasound diagnostic apparatus, an ultrasonic probe is moved in a perpendicular direction to a scanning surface to obtain three-dimensional data reflecting a three-dimensional structure, and a three-dimensional image is formed and displayed as if the three-dimensional structure were viewed from a desired line of sight direction.
FIG. 1 is a schematic view of three-dimensional data obtained by scanning a fetus F in a mother""s body by an ultrasound diagnostic apparatus.
When it is attempted to form a three-dimensional image of the fetus F from the data of FIG. 1, due to the presence of a uterus wall W outside the fetus F, the uterus wall W constitutes an obstacle so that a three-dimensional image of the fetus F cannot be formed. Hence, the operator sets a region of interest ROI, and forms a three-dimensional image so that structures outside the region of interest ROI cannot be seen, as shown in FIG. 2.
However, if part of the fetus F is in contact with the uterus wall W, it is impossible to distinguish the boundary as the structural consistencies of the fetus F and uterus wall W are substantially identical, so the operator cannot set a region of interest ROI correctly, as shown in FIG. 3.
It is therefore a first object of this invention to provide a boundary detecting apparatus which can detect a boundary line of different objects in contact which appear to be the same object as their structural consistencies are identical or substantially identical, and to provide an image processing apparatus which can obtain a three-dimensional image of a fetus without obstruction by the uterus wall in contact with the fetus.
FIG. 4 is a schematic view of three-dimensional data obtained by scanning a liver L with invasive cancer C by an ultrasound diagnostic apparatus.
The liver L is covered by a peritoneum P, and as the two have different structural consistencies, the surface of the liver L can be extracted based on the difference of structural consistencies and a three-dimensional image of the surface Ls of the liver can be formed, as shown in FIG. 5.
However, in this three-dimensional image, it was impossible to distinguish the part with the invasive cancer C adhering to the peritoneum P.
It is therefore a second object of this invention to provide a non-boundary line detecting apparatus which can detect a non-boundary line of different objects in contact which appear to have a boundary as their structural consistencies are largely different, but which are actually part of the same structure, and to provide an image processing apparatus which can depict a liver such that invasive cancer on the liver surface can be identified.
In a first aspect, this invention provides a boundary line detecting apparatus comprising a motion vector detecting means which calculates motion vectors of plural parts in a first or second image from a first image obtained by imaging at a first time and from a second image obtained by imaging at a second time, a boundary part extracting means which extracts a moving boundary part from the difference between the motion vectors of the plural parts, and a boundary line estimating means which estimates a boundary line between objects with different motion based on the extracted boundary part.
Even if different objects in contact appear to be the same object because their structural consistencies are identical or substantially identical, if the different objects are not part of the same structure, a difference in the motion of the two arises in many cases.
Therefore, in the boundary line detecting apparatus of the first aspect, motion vectors in plural parts of the image are found from plural images obtained at different times, a boundary part is extracted from this difference, and a boundary line between the different objects is estimated from this boundary part. In this way, the boundary line between different objects in contact which appear to be the same object because their structural consistencies are identical or substantially identical, can be detected.
In the above construction, the plural parts may for example be obvious parts such as speckles (granular noise), points on a virtual lattice on the image, or all of the picture elements of the image.
In the aforesaid construction, the boundary line may be estimated by finding the straight line or curve which best fits the boundary part, e.g. by the method of least squares.
The motion of the objects may be an automatic motion such as a respiration or pulse, or it may be forcibly produced by applying an external force (e.g., pressing hard with the ultrasonic probe).
In a second aspect, this invention provides an image processing apparatus which forms an image based on data reflecting a two-dimensional structure, comprising a boundary line detecting means which detects a boundary line in the two-dimensional structure using the boundary line detecting apparatus of the first aspect, and a filter means which forms or displays an image so that the two sides of the boundary line can be distinguished when the image is formed or displayed.
In the image processing apparatus of the second aspect, an image is formed or displayed so that the two sides of the boundary line detected using the boundary line detecting apparatus of the first aspect can be distinguished, so a region of interest can be automatically set or easily set by an operator so as to exclude one of the objects on either side of the boundary line.
In a third aspect, this invention provides an image processing apparatus which forms a three-dimensional image when a three-dimensional structure is viewed from a desired line of site direction based on data reflecting the three-dimensional structure, comprising a boundary surface detecting means which detects a boundary surface in the three-dimensional structure using the boundary line detecting apparatus of the first aspect, and a filter means which forms or displays a three-dimensional image such that the three-dimensional structure which is in front of the boundary surface in the line of sight direction cannot be seen when the three-dimensional image is formed or displayed.
In the image processing apparatus of the third aspect, a boundary surface is detected using the boundary line detecting apparatus of the first aspect on data reflecting a three-dimensional structure, and a three-dimensional image is formed or displayed such that the three-dimensional structure which is in front of the boundary surface in the line of site direction cannot be seen. Therefore, a three-dimensional image of, for example, a fetus can be obtained without interference from the uterus wall in contact with the fetus.
In a fourth aspect, this invention provides a non-boundary line detecting apparatus comprising an object contour line extracting means which extracts a contour line of an object in a two-dimensional structure based on data reflecting the two-dimensional structure, and a non-boundary line detecting means which detects a part which is a contour line of the object but not a boundary line as a non-boundary line using the boundary line detecting apparatus of the first aspect.
If a part of the imaging object is integrated with a surrounding object, no difference emerges in the motion of the two in this part, so if the part where there is no difference of motion is situated on the contour line of the imaging object, it is understood that the two are integrated in this part.
Therefore, in the non-boundary line detecting apparatus of the fourth aspect, motion vectors of plural parts in an image are found from plural images obtained at different times, the boundary line is extracted from their difference, and a part of the object which is a contour line but not a boundary line is detected as a non-boundary line. In this way, a non-boundary line is detected in different objects in contact which appear to have a boundary line as their structural consistencies are largely different, but which are actually part of the same structure.
In a fifth aspect, this invention provides an image processing apparatus, comprising an object surface extracting means which extracts the surface of an object contained in a three-dimensional structure based on data reflecting the three-dimensional structure, a non-boundary surface detecting means which detects a part which is a surface of an object but not a boundary surface as a non-boundary surface using the non-boundary line detecting apparatus of the fourth aspect on the data, and an object surface/non-boundary surface image forming means which forms and displays an image or three-dimensional image of the surface and non-boundary surface of the object.
In the image forming apparatus of the fifth aspect, a non-boundary surface is detected using the non-boundary line detecting apparatus of the fourth aspect on data reflecting a three-dimensional image, and an image or three-dimensional image of the surface and non-boundary surface of the object is formed or displayed. Therefore, for example, a liver can be depicted so as to be able to identify invasive cancer on the liver surface.
From another aspect, this invention provides a boundary line detecting method wherein motion vectors of plural parts in an image are found from plural images obtained at different times, a boundary part is extracted from their difference, and a boundary line is estimated from this boundary part.
From another aspect, this invention provides an image processing method which forms or displays an image such that two sides of a boundary line detected using the boundary line detecting method, can be identified.
From another aspect, this invention provides an image processing method which detects a boundary surface using the boundary line detecting method on data reflecting a three-dimensional structure, and forms or displays a three-dimensional image such that the three-dimensional structure which is in front in the line of sight direction from the boundary surface cannot be seen.
From another aspect, this invention provides a non-boundary line detecting method wherein motion vectors of plural parts in an image are found from plural images obtained at different times, a boundary part is extracted from their difference, and a part which is not the boundary part but which is a contour line of the object is detected as a non-boundary line.
From yet another aspect, this invention provides an image processing method which detects a non-boundary surface using the non-boundary line detecting method on data reflecting a. three-dimensional structure, and forms or displays an image or three-dimensional image of the surface and non-boundary surface of the object.
According to the boundary line detecting apparatus and image processing apparatus of this invention, a boundary line can be detected between different objects in contact which appear to be same object as their structural consistencies are identical or substantially identical, and a three-dimensional image of a fetus can be depicted without interference from a uterus wall in contact with the fetus.
Further, according to the non-boundary line detecting apparatus and image processing apparatus of this invention, a non-boundary line can be detected between different objects in contact which appear to have a boundary as their structural consistencies are largely different, but which are actually part of the same structure, and a liver can be detected so that invasive cancer on the liver surface can be identified.